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ABSTRACT 



Interviews with teams of teachers at two Australian 
technical and further education (TAFE) institutes (the Lawson Institute of 
TAFE Motor Mechanics and the Franklin Institute of TAFE Disability and Aged 
Care) were used to examine the theoretical and practical tensions experienced 
by TAFE teachers when making assessment judgments within the original 
competency-based training paradigm and according to the assessment criteria 
incorporated into training packages. Both teams stated that assessment 
judgments are not simple, consistent, reliable, or reproducible. The study 
established that individuals and teams were making judgments within a social 
and historical context and that a range of problems must be solved during the 
judgment process. The Lawson staff freely acknowledged that individuals in 
the team apply differing standards when assessing students. Team members 
appeared proud of their own professional standards, and most believed that 
their practice of assessing students differently prepares students for the 
"real world, " where standards vary markedly. The Franklin staff members were 
distinctly committed to being a strong team that consciously works through 
issues together and seeks to have a collective response. The unit of judgment 
at Lawson was the individual teacher, whereas at Franklin, the unit of 
judgment was the team. (Contains 11 references.) (MN) 
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Judgement has become a significant area of interest in philosophy, rhetoric and 
popular culture (Beckett 2000; Hager 2000; Smith 1999). The need to make a 
'judgement call' is often acknowledged with a sense of pride, almost as an 
affirmation of the worth of the speaker, an antidote to the alienation of the individual 
in the postmodern world (Sloop 1998). What then of the experience of technical and 
further education (TAFE) teachers in making assessment judgements in the post- 
curriculum age? Do they experience assessment judgements as autonomous and 
professionally empowering events? How do we prepare teachers to make good 
judgements? What sort of professional practices sustain wise judgements? This paper 
will explore these questions by providing some glimpses into the judgement 
experiences of two teams of TAFE teachers. 



This study deals with the theoretical and practical tensions experienced by teachers 
making assessment judgements within the original competency-based training (CBT) 
paradigm and according to the assessment criteria incorporated into Training 
Packages 1 . On a theoretical level there has been widespread debate regarding the 
essential ambiguity or otherwise of documented units of competence and assessment 
criteria (Jones 2000). Numerous authors have indicated that the Training Packages 
and the associated Assessment Criteria provide 'thin' frameworks for delivering 
courses and making assessments of skill, let alone knowledge and understanding 
(Mulcahy 2000). The raison d'etre of national Training Packages is to ensure that 
valid, reliable and reproducible assessments are made in industry and on campus 
from Broome to Burnie. This is clear in the following extract from the ANTA 2 
website. 



Why use a Training Package? 

Training Packages provide your employees with qualifications which are 
recognised nationally. This means an individual who gained their 
qualification in Cairns has the same qualifications as a person who trained 
in Ballarat. A recognised qualification is therefore a guarantee that the 
skills of the employee (or potential employee) are: 

• formally recognised 

• consistent nationally across the industry or industry sector. 
(ANTA website: http://www.anta.gov.au/tp/ ) 
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The implicit value of 'consistency', particularly in assessment, is constantly 
emphasised in Australian National Training Authority (ANTA) documentation (see 
Appendix One). The concept of national qualifications both depends on, and 
promises to deliver, this notion of consistency. 



However at a practical level, teachers work in the '. . . gulf between precept and 
practice . . / (Brown 2000). They navigate between the 'tick and flick' mythology 
surrounding competency based assessment versus the reality that CBA 3 requires a 
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high degree of interpretation and judgement'. 4 The roadmaps provided to 
teachers, the documented competency standards and assessment criteria, equip them 
with very thin descriptions of the assessment judgement process. On the other hand, 
the lived experience of making assessment is a thick, social experience. 
Organisational and personal history and culture, the teacher's 'community of 
practice 7 (Wenger 1998), all provide a context within which teachers make 
judgements. Consequently assessment judgements vary between individual 
assessors within providers, let alone between providers and across state borders. 

This paper will explore assessment judgements made within two 'communities of 
practice' or groups of teachers to illustrate and tease out some of these issues. 

Methodology 

The interviews reported here are part of a larger group of long, semi-structured 
interviews with teachers and assessors in public and private providers and in 
industry. The interviews were transcribed and subjected to reflective and critical 
analysis to develop working hypotheses on the nature of vocational education and 
training (VET) practice in making assessment judgements. Analysis of six initial 
interviews suggested several themes that seemed worthy of further investigation. 

The social context in which assessment judgements were made appeared to be a 
particularly important factor informing decision making and assessment of 
competence. To further explore this theme, subsequent interviews were conducted 
with clusters of participants who worked together. Several interviews were also 
conducted with participants who, by the nature of their work, were forced to make 
assessment judgements in professionally isolated situations. In this paper we shall 
examine the stories of two teams: the Lawson Institute of TAFE Motor Mechanics 
teachers and the Franklin Institute of TAFE Disability and Aged Care teachers. 

Lawson Institute of TAFE Motor Mechanics team 

The Lawson Motor Mechanics team is located on a campus that was formerly a stand 
alone, mono-purpose college. It is situated in a manufacturing area with high a 
migrant population and high youth unemployment. The staff members interviewed 
were bitter about an enforced amalgamation, five years previously, which they 
regard as a takeover by Lawson... 

Our formal meetings ... which we have about once every three weeks ... 
we get together and have a big fat whinge generally at management, 
about how they’re not doing their job, and how if they were doing their 
job, our job would be a lot easier. And because we’ve been taken over by 
Lawson we seem ... personally I call it ...we are the little brother of the 
relationship ... and it all happens up the road, nothing gets done down 
here ... We believe that head office 5 is taking all the cash . . . Shane 

The mood is cynical and depressed. Whilst I was interviewing, an administrative 
officer came around to do a room check for PETE 6 room occupancy statistics, eliciting 
instant resentment. Student numbers have been constantly falling in recent years, 
resources have been cut back and each informant alluded to literacy as a huge barrier 
to learning. 



... and there was two heads of department and three or four SDAs 7 . In the 
last eight years that’s dropped from two heads of department down to one 
program coordinator. We’ve gone from 22 to probably 13 staff ... 8,10, 6, 
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probably 14 staff. So the number of staff have dropped, the number of 
apprentices have dropped from 110, 120 I suppose, down to motor 
mechanics have 48, the electricians have twelve, that’s 60, so there’s been 
a great drop. We’ve lost, over the last two months; we’ve lost two extra 
staff, because of natural attrition. One guy took a VDP 8 . Shane 

The campus is still basically a mono-purpose automotive facility at a distance from 
the multi-disciplinary main campus. The staff tend only to have regular contact with 
each other and their students. They don't see much of management who are mostly 
on the main campus. They teach in adjacent classrooms or team-teach in one room; 
they have regular 'smoko' and lunch breaks together sitting around one large table in 
a staff room. Many of them have done this for more than ten years whilst their 
numbers and the numbers of students have shrunk. 

Our institute from our point of view is money driven. Get the people in, 
get the money in, so we can show a profit at the end of the year. Whether 
it’s good or not so good for the apprentices is debatable but the managers 
say yeah you’ve done a good job. We’ve made hundreds of thousands of 
dollars of profit, so fantastic. Yet we feel it’s a sausage factory, sausages 
in, sausages out. Shane 

Franklin Institute of TAFE Disability and Aged Care team 

The Franklin Institute City Campus team was suggested as an example of a team 
with a strong commitment to professional practice. Franklin has five campuses 
formed by a series of amalgamations and restructures. The City Campus Disability 
and Aged Care team is about to be reorganised as part of yet another restructure. The 
staff will remain physically close in a shared office but the two courses they teach 
will be allocated to different departments. Most of them already teach small loads in 
other departments - now everyone's load will be spread. Their office is on the third 
floor of a busy multi-storey building located near a fashionable retail precinct. The 
interior of the building is painted traditional 'TAFE' grey, but there are cheerful 
pictures and ornaments in the offices. 

The team consists of three contract teachers: Felicia, Maggie and Mercedes, and one 
sessional teacher (not interviewed) and their coordinator, Jane, who is about to leave 
teaching for a management job in human services. Felicia, Maggie and Jane have 
worked together for a long time: five years or more. The Certificate III in Community 
Services (Aged Care Work) that they teach together is also taught on another 
Franklin Campus, Rural Campus, located about 30 kilometres away. 

... it means that that team of people will be split ... reasonably 
significantly. Disability stays here and Aged Care goes, and people’s 
positions will be sort of half here and half there, so ... yeah, it will be 
interesting to see how they work team wise. I think geographically 
people will stay together, but it’s really about where your allegiance is 
departmentally, and I think that can be quite fragmented, fragmenting 
really. Jane 



Consistency in making assessment judgements 

For both teams assessment judgements are not simple, nor are they consistent, 
reliable or reproducible. It is clear that individuals and teams make judgements 
within a social and historical context and that a range of problems need to be solved 
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during the judgement process. The experience is thick and socially redolent, The act 
of making an assessment judgement also has ethical and tacit dimensions forming 
the gel in which the whole judgement process is solidified for the practitioner. 
Members of the Motor Mechanics team at Lawson demonstrated a kind of rugged 
individualism in their attitude to making assessment judgements. They freely 
acknowledge that individuals in the team apply differing standards when assessing 
students. Not only is each member proud of his/her own professional standard but 
most told me that their practice of assessing differentially prepares students for the 
"real world', where standards also vary markedly. 

The teacher who, not normally teaches there, signs off ... completes that 
module with the kid. There’s very little time, very little opportunity, for 
me to go back and retest those students. There is no need, there is no 
place in our system for it. So only the kid that’s finished half a job will I 
retest or go back and have a look. So a lot of kids are now realising that 
Mr Lauder’s in the class, we’d better finish it all before he comes in. 

They go ‘are you in the next session?’ They go you beauty because I set 
the test a little bit harder. Shane 

Well, we’ve discussed it, but we’ve all got different standards ... some 
say nah ... I’ve had a student working for three days and they must know 
the entire... Darren 

The day you walk through the door I start to judge you, whether you like 
it or not you will be judged by me, and to my standard. Now whether 
that’s a right thing or a wrong thing to do, somebody somewhere along 
the line is going to ring up and say what do I think of you? And I can 
only judge you on how you behave in this room . . . Bruce 

All the teachers are different and maybe some are a little bit soft and 
maybe some are a little bit too hard, but then again, that’s the nature of 
where they’re going to work in the real world too. They’re going to run 
against people who are hard ... and fussy and they’re going to come 
across people where the other person’s more mild and a little more ... 
that, provided that it’s not too far the other way and the person’s just 
marking them off for nothing - and that’s happened . . . Bruce 

The Franklin staff are distinctly committed to being a strong team that consciously 
works through issues together and seeks to have a collective response. It is very 
important to these teachers that they agree on decisions like assessment judgements. 
The team discusses differences of opinion and individuals are prepared to make 
considerable compromises in order to reach consensus. Their assessment judgements 
are consistent within the team. 

Well I can’t remember a time when we didn’t actually work something 
out ...I mean I might not have been altogether OK about it or someone 
else may not have, but I think that we actually come to some conclusion 
about it, yeah, but we agree really. . . Felicia 

In comparison with Lawson where the unit of judgement is the individual teacher, at 
Franklin the unit of judgement is the team. At Lawson, inconsistency in making 
assessment judgements occurred between individuals; at Franklin it occurs between 
teams. The Franklin City Campus Team is aware that the content and underpinning 
philosophy of the Certificate III in Community Services (Aged Care Work) as they 
teach it differs from the approach used by their colleagues at the Franklin Rural 



Campus. The City team follow a community-based model of Aged Care, whilst the 
Rural team support a clinical model. This difference in approach originated in the 
differing employment histories of team members on the two campuses. The teachers 
in the City team come largely from a community development background, whilst 
the members of the Rural team have been nurses and other paramedical workers. 

... look I think a lot of it is about the original group of people whoever 
they were I suppose, and who we then employ and we tend to employ 
people to match our original group. So it would be unlikely, just as an 
example, the Aged Care area here in this area tends to have a bit more of 
a community based approach to aged services in some of the other areas, 
you know even other colleagues in another campus tend to take a bit more 
of a medical model approach because they’re primarily the nurses who 
worked in that area ... so we would be reluctant to or unlikely to get 
someone who has a strongly medical model approach to Aged Care to 
work in our area, ‘cause its not the approach that we take so it’s ... you 
know, wherever that started I suppose ... Jane 

Despite the amalgamation that brought these two teams into the one institution, 
there has not yet been any attempt to resolve the differing educational approaches of 
the two groups. This occurs to the extent that they acknowledge that assessment 
judgements may differ between the two teams of teachers. 

I guess the sorts of things we would want to come out of one of the Aged 
Care modules is that people would need to have an understanding of the 
broader sociological implications of aging I guess, and that doesn’t need 
to be on any deep academic level but that broader thing of where ageism 
fits in. Why there is ageism. Why older people are treated the way they 
are, so that from our perspective we think that that’s important that people 
know that so that when they work with people in a nursing home they can 
actually have an understanding of all this marginalised group of people 
and they are being removed and why is this here and why are people 
treated in that way, there’s restraint and all sorts of things that happen to 
people and that’s not just about ... maybe the [Rural Campus] people 
may well talk about the medical ... 

about how inappropriate that was and that medically the correct treatment 
would be to do this, this and this, and how that was really poor practice 
... They’re absolutely right and I think both things are right, but in terms 
of the assessment I suppose, we would use a different assessment process. 

Jane 

On being asked about the impact of a national curriculum on their work one team 
member responded: We haven't even got a state one and zue haven't even got an 
institutional one really ! 

The plot thickens 

For teachers, the professional social context or community of practice in which they 
make an assessment judgement is clearly a significant determinant of the outcome of 
the judgement process. Many other factors interplay with this social context 
including tacit experiences and ethical concerns. A few examples follow: 

It’s been signed off by the trainee and his ... employer and there’s just no 
way where that the equipment is even available in the workshop, yet 
they’ve been signed off as to be competent. Some examples have been in 
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brake and underbody where the managers says that the person is 
competent to do brake drum machining and they don’t even have that 
equipment in the workshop. So its not ... its not an isolated incidence, 
but you know, it is a problem with one where clearly they haven’t had the 
opportunity to work on. Darren 

... the sort of manager of this big organisation had resigned and other 
person came in their place and I guess this person’s agenda was more 
about ... ensuring the organisation coming in on budget. So almost the 
first thing she had done was cut back people’s hours, pays, a whole range 
of things, so staff then were involved in pretty significant industrial 
action. And that all happened in the middle of this assessment process. 
And that really coloured ... how we assessed because the individuals we 
spoke to were reluctant to say anything positive at all about the 
organisation which meant that they didn’t want to talk about any training 
they’d done, they just . . . they were really concerned about that. Jane 

... and you can identify that a person is competent by the way they go 
about their work, you cannot assess every ... every competency that the 
employer has said that person is competent with. Darren 



Implications for national training system 

This research points to a gulf between the rhetoric of the Training Reform Agenda 
and teaching practice. The experiences of the teachers at Lawson and Franklin 
suggest that if we want consistency we are going to have to do more than mandate 
for it. Where teams of teachers were interviewed, it seemed that not only the present 
social context but also the history of the team impinged on assessment decisions. The 
team history, moreover, had unfolded in the greater social and political story of the 
provider and the system in which it operates. John Seely Brown made an apt 
comment when he said: 

Many organisations are willing to assume that complex tasks can be 
successfully mapped onto a set of simple, Tayloristic, canonical steps that 
can be followed without need of significant understanding or insight (and 
thus without need of significant investment in training ... 

(Brown 2000) 

The roadmaps supplied with Training Packages do not take account of the rich 
historical, social and ethical context in which real teachers and assessors make 
judgements. The act of judgement is not a simplistic instrumental activity but an 
ethical, attentive process (Smith 1999). If we do value consistency to any degree we 
are going to have to supply much thicker documents to guide assessors. They will 
need professional development and encouragement to form the vigorous 
communities that are needed to normalise judgements. We will also have to address 
system-wide structural problems like leadership and change management if being a 
valued team member is a precondition for making consistent judgements. At last 
year's AVETRA conference, Robin Booth rightly pointed out that there are no simple 
answers to improving consistency in assessment (Booth 2000). 

We also need to consider the possibility that inconsistency has a place in making 
assessment judgements. We have only to consider the dissension that takes place in 
the High Court to realise that highly trained and experienced professionals do make 
different decisions. The individuals at Lawson and the City Campus team at Franklin 
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knowingly took pride in making decisions that varied from those of their colleagues. 
We can view this behaviour as dysfunctional or we can see it as an extreme 
expression of professional autonomy that can also become a positive force in 
teaching and learning. Much previous research (Jones 2000) has shown that the 
perceived loss of autonomy has been seen as a problem by teachers adjusting 
successively to competency based assessment, national curriculum and Training 
Packages. Autonomy has been identified as an important condition for teacher 
commitment and, therefore, for effective teaching (Firestone 1993). 

The National Training System has aligned with a prescriptive rather than a more 
individual, evolutionary approach to making assessment judgements. There is clear 
mismatch at present between the thinness of this approach as documented in the 
Training Package assessment criteria and the thickness of the learning environments 
in which assessment judgements are made. This dilemma needs to resolved at a 
national and state training system level. The resolution will not be easy because there 
are a whole raft of political questions involved in the problem. Whatever the 
solution, learners will only benefit from effective assessment judgements where there 
is some nurturance of professionalism in teachers and teaching communities. 



Appendix one: excerpt from National Assessment Principles 

These principles were endorsed by the Ministerial Council of the Australian National 
Training Authority on 22 May 1998: 

http://www.anta.gov.au/PUBS/ARF ARRANGEMENTS/ antaass.pdf . 



Principle 


Essential features 


8. Assessment 
processes shall 
be valid \ 
reliable, flexible 
and fair 


8.1 Assessment processes must cover the broad range of skills 
and knowledge needed to demonstrate competency. 

8.2 Assessment of competence should be a process which 
integrates knowledge and skills with their practical 
application. 

8.3 During assessment, judgements to determine an 
individual’s competence should, wherever practicable, 
be made on evidence gathered on a number of 
occasions and in a variety of contexts or situations. 

8.4 Assessment processes should be monitored and reviewed 
to ensure that there is consistency in the interpretation 

of evidence. 

8.5 Assessment should cover both the on- and off-the-job 
components of training. 

8.6 Assessment processes should provide for the recognition of 
competencies no matter how, where or when they have 

been acquired. 

8.7 Assessment processes should be made accessible to 
individuals so that they can proceed readily from one 
competency standard unit to another. 

8.8 Assessment practices must be equitable for all groups or 
individuals. 

8.9 Assessment procedures and the criteria forjudging 
performance must be made clear to all individuals 
seeking assessment. 

8.10 There should be a participatory approach to assessment. 
The process of assessment should be jointly developed/ 
agreed between the assessor and the assessee. 

8.1 1 Opportunities must be provided to allow individuals to 
challenge assessment decisions, with provision for 
reassessment. 



Notes 



1. The phrase Training Package is by convention capitalised when used 
to denote the Training Package system established by the Australian 
National Training Authority as opposed to all other applications of 
this term. 

2. ANTA = Australian National Training Authority. 

3. CBA = competency based assessment. 

4. Bull (1985) suggested that '...competency-based vocational education 
may work well when applied to tasks which tend to be repetitive - ie. 
those in which interpretation and judgement play a relatively small 
part' . 

5. 'Head office' refers to the main Lawson campus. 

6. Victorian Office of Post-Compulsory Education and Training. 

7. SDA refers to 'senior duties allowance', an old promotion position in 
the Victorian TAFE system. 

8. Voluntary Departure Package. 
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